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ABSTRACT 

— i 

Two  alternative  methods  for  optimizing  an  unconstrained  non-linear 
function  are  investigated  and  compared.  The  investigations  are  made 
subject  to  a  restriction  as  to  the  number  of  function  evaluations 
available  to  conduct  the  optimization  procedures.  Powell's  method  of 
conjugate  directions  is  employed  as  the  direct  search  method  and  is 
considered  the  reference  method.  The  alternate  method  is  based  on 
fitting  a  quadratic  surface  to  the  available  function  evaluations  and 
optimizing  over  the  resulting  fitted  surface.  The  test  functions 
considered  in  the  investigation  were  limited  to  unimodal  functions. 
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I.  NATURE  OF  THE  PROBLEM 

A.   INTRODUCTION 

In  Operations  Research  a  major  problem  concerns  the  determination  of 
the  optimum  response  for  an  objective  function.   In  many  cases,  the 
objective  function  may  not  lend  itself  to  expression  in  closed  analytical 
form.  Information  concerning  the  nature  of  the  objective  function  may 
only  be  available  through  computer  simulation,  field  testing  or  a  combina- 
tion of  both.  Furthermore,  restrictions  may  exist  with  respect  to  the 
quantity  of  information  available  or  obtainable  due  to  the  costs  involved 
in  securing  the  data.  Thus,  it  is  incumbent  that  the  most  efficient  use 
be  made  of  the  limited  information  available  in  order  to  achieve  the  best 
possible  results  in  determining  the  optimum  response. 

There  exist  several  search  techniques  that  could  be  adapted  to  solve 
the  problem  as  stated  above.  Generally,  these  search  techniques  could 
be  classified  as  direct  search  techniques  or  gradient  search  techniques. 
Direct  search  methods  do  not  require  the  computation  of  partial  deriva- 
tives of  the  objective  function.  Determination  of  the  optimum  response 
is  based  solely  on  values  of  the  objective  function  itself.  On  the  other 
hand,  gradient  search  methods  compute  values  of  partial  derivatives  of 
the  objective  function.  These  resulting  values  are  then  used  in  select- 
ing future  search  directions.  Search  techniques  employing  gradient 
methods  require  more  information  (i.e.,  additional  function  evaluations, 
computer  simulations,  etc.)  than  direct  search  procedures.  Thus,  with 
the  presence  of  restrictions  on  the  quantity  of  information  available  it 
would  be  more  advantageous  to  use  a  direct  search  method. 


Direct  Search  methods  can  be  broadly  divided  into  three  classifica- 
tions: tabulation  methods,  sequential  methods  and  linear  methods. 
Tabulation  methods,  as  the  name  implies,  result  in  a  tabular  listing  of 
objective  function  responses  for  different  values  of  the  input  variables. 
The  "optimum"  response  is  considered  to  be  the  "best"  value  found  in  the 
table.   In  the  case  of  minimization  problems,  the  "best"  value  would  be 
the  smallest  value.  An  example  of  a  tabulation  method  is  random  search. 

Sequential  methods  employ  the  use  of  geometric  designs  in  the  input 
variable  space.  The  objective  function  is  evaluated  at  the  vertices  of 
the  geometric  design.  The  sequence  is  repeated  until  the  desired  accuracy 
is  achieved  or  the  restriction  of  the  quantity  of  information  obtainable 
precludes  additional  testing.  Factorial  designs  and  the  simplex  method 
are  examples  of  sequential  methods. 

Linear  methods  employ  the  use  of  a  set  of  direction  vectors  through- 
out the  conduct  of  the  search.  The  objective  function  is  evaluated  at 
different  points  along  these  search  directions  and  conduct  of  the  search 
is  dependent  upon  the  results  obtained.  The  set  of  direction  vectors 
may  or  may  not  be  changed  during  the  course  of  the  search.  Examples  of 
linear  methods  are  the  alternating  variable  search  and  Powell's  method 
of  conjugate  directions. 

An  alternative  approach  to  employing  search  techniques  to  solve  the 
problem  would  be  to  use  curve-fitting  techniques.  Using  this  technique, 
test  points  would  be  established  in  the  input  variable  space  and  the 
objective  function  would  be  evaluated  at  these  test  points.  A  least- 
square  polynomial  surface  would  then  be  fitted  to  these  resulting 
objective  function  values.  To  complete  the  procedure  the  location  of 
the  "optimum"  response  of  the  fitted  surface  would  be  determined  and  the 


resulting  input  variable  values  would  be  input  to  the  true  objective 
function  to  determine  the  "optimum"  response. 

B.  STATEMENT  OF  THE  PROBLEM 

The  specific  problem  that  was  the  subject  of  investigation  was  to 
compare  the  relative  efficiency  of  a  curve-fitting  technique  as  opposed 
to  a  search  technique.  The  assumption  was  made  that  a  restriction  exists 
on  the  quantity  of  information  available  (i.e.,  there  were  a  finite  number 
of  functional  evaluation  that  could  have  been  made).  A  direct  result  of 
the  above  assumption  was  that  the  investigation  was  limited  to  those 
techniques  which  do  not  require  the  computation  of  partial  derivatives. 
Furthermore,  the  investigation  was  restricted  to  test  objective  functions 
of  a  unimodal  nature  insuring  the  existence  of  only  one  true  optimum 
response. 


II.  EXPERIMENTAL  PROCEDURES 

A.  SEARCH  PROCEDURE 

Powell's  method  of  conjugate  directions  was  chosen  as  the  direct 
search  technique  to  be  used.  Powell's  method  is  considered  to  be  one 
of  the  most  efficient  direct  search  techniques  available  [Ref.l].  Its 
efficiency  is  exceptionally  good  for  functions  that  can  be  approximated 
by  a  quadratic  in  the  region  of  the  optimum. 

Powell's  method  commences  with  a  search  along  a  set  of  linearly 
independent  directions  that  span  the  input  variable  space  starting  from 
an  arbitrarily  chosen  initial  point.  The  initial  set  of  directions 
chosen  are  the  co-ordinate  directions.  At  the  completion  of  each  itera- 
tion a  new  direction  is. defined  and  replaces  one  of  the  presently  exist- 
ing directions  only  if  the  new  set  of  directions  is  at  least  as  efficient 
as  the  present  set.  If  this  is  not  the  case,  an  additional  iteration  is 
conducted  retaining  the  present  set  of  vectors.  A  complete  description 
of  Powell's  method  and  a  detailed  discussion  of  its  efficiency  and 
convergence  is  contained  in  [Ref.2]. 

An  existing  version  of  Powell's  method  programmed  in  FORTRAN  for  use 
on  the  IBM  360  Computer  was  used  as  a  subroutine  in  the  overall  computer 
program  written  to  conduct  the  search  technique  optimization.  The  main 
program  provided  the  calling  argument  required  by  the  subroutine  to 
conduct  the  optimization. 

B.  CURVE-FITTING  TECHNIQUE 

The  curve-fitting  technique  used  was  based  on  standard  multiple 
linear  regression  techniques  [Refs.  3  and  4]  modified  as  follows. 
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Because  the  test  functions  to  be  investigated  were  unimodal  the  decision 
was  made  to  fit  a  quadratic  surface  to  the  experimental  data.  Thus,  the 
order  of  the  regression  was  double  the  number  of  input  variables. 

An  early  decision  required  to  be  made  concerned  exactly  what  proced- 
ure would  be  used  to  place  the  test  points  in  the  input  variable  space. 
Two  possibilities  were  considered.  A  completely  random  selection  of  the 
test  points  was  one  alternative.  A  second  approach  was  to  place  the  test 
points  throughout  the  space  in  accordance  with  an  experimental  design. 
The  first  experimental  design  considered  was  a  composite  factorial  design 
[Ref.  5].  This  approach  was  attractive  for  objective  functions  of  few 
input  variables.  However,  the  requirement  for  an  excessive  number  of 
test  points  and  consequently  an  increased  number  of  function  evaluations 
precluded  the  use  of  this  design  for  objective  functions  of  more  than 
four  variables. 

For  functions  of  more  than  four  variables  it  was  decided  to  use  an 
orthogonal  design  requiring  2n+l  test  points,  where  n  is  the  number  of 
input  variables.  The  initial  test  point  was  placed  in  the  center  of  the 
input  variable  space  and  the  remaining  points  were  placed  at  a  fixed 
distance  (0.5)  in  both  the  positive  and  negative  direction  along  the 
co-ordinate  axes  passing  through  the  center  of  the  design.  In  the 
investigations  a  third  alternative  was  also  considered.  For  those  cases 
where  the  number  of  observations  (function  evaluations)  available  exceeded 
the  number  required  for  the  experimental  design,  the  remaining  points 
were  located  randomly  in  the  input  variable  space. 

Once  the  test  points  were  placed,  the  function  was  evaluated  at  these 
points.  The  value  of  input  variables,  the  squares  of  the  input  variables 
and  the  value  of  the  objective  function  for  each  test  point  were  then 


input  to  the  regression  subroutine  to  determine  the  regression  co- 
efficients of  the  fitted  surface.  The  resulting  fitted  surface  was  a 
quadratic  of  the  form: 

n         n     2 
f  =  E  a-  x-  +  I     b.  x-  +  constant  term 
i=l  1  7   i=l  1     n 

Provided  the  quadratic  was  positive  (negative)  definite  an  explicit 

determination  of  thp  minimum  (maximum)  value  of  the  quadratic  could  be 

computed.  In  that  case,  the  minimum  (maximum)  value  of  the  quadratic 

occurs  at  the  point: 

1  a' 
x  •  =  -  2"  r   )   l  =  I  ,  . . .  ,  n 

These  values  of  x-  were  computed  and  substituted  into  the  true  objective 
function  to  determine  the  "optimal"  objective  function  response.- 

Preliminary  results  indicated  that  the  curve-fitting  technique  was 
generating  a  large  number  of  indefinite  quadratic  surfaces.  This  situa- 
tion occurred  more  commonly  when  random  choice  was  used  to  locate  the 
test  points.  Any  indefinite  quadratics  generated  were  not  useful  because 
it  was  not  possible  to  find  the  location  of  the  minimum  response  of  the 
fitted  surface.  Thus,  an  "optimum"  response  could  not  be  determined  in 
those  situations  where  the  curve-fitting  technique  generated  an  indefinite 
quadratic. 

In  an  effort  to  overcome  this  problem,  a  procedure  was  introduced 
into  the  curve-fitting  technique  to  reduce  the  number  of  observations 
considered  by  the  regression  procedure.  The  basic  idea  behind  the  pro- 
cedure was  for  the  regression  procedure  to  restrict  its  consideration 
to  those  test  points  that  were  close  to  the  optimum.  In  that  manner 
points  at  a  distance  from  the  optimum  did  not  affect  the  fit.  It  was 
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felt  that  this  procedure  would  insure  a  better  fit  in  the  area  of  the 
optimum  and  thereby,  eliminate  the  number  of  indefinite  quadratics 
generated. 

A  search  was  made  of  the  objective  function  values  for  each  test 
point.  The  worst  25  percent  of  the  test  points  were  then  discarded 
reducing  the  number  of  observations  considered  by  the  regression  pro- 
cedure. Because  the  curve-fitting  procedure  was  designed  to  find  the 
minimum  objective  response,  the  worst  25  percent  of  the  test  points  were 
considered  to  be  the  25  percent  with  the  highest  values  of  the  objective 
function.  This  procedure  was  then  repeated  eliminating  the  worst  50 
percent  of  the  test  points. 

This  entire  procedure  was  programmed  in  FORTRAN  for  use  on  the  IBM 
360  computer.  The  program  as  written  is  capable  of  handling  up  to  20 
input  variables  and  50  observations  of  test  points.  A  detailed 
description  of  the  curve-fitting  program  is  contained  in  Appendix  A. 

C.  TEST  FUNCTIONS 

Two  types  of  objective  functions  were  considered  in  the  investigation; 
quadratic  functions,  with  cross  product  terms,  and  exponential  functions. 
The  decision  to  include  cross  product  terms  in  the  quadratic  functions 
was  made  to  preclude  the  possibility  of  achieving  an  exact  fit  when 
employing  the  curve-fitting  technique.   In  the  case  of  exponential  func- 
tions, the  exponent  term  was  a  quadratic  function  chosen  so  that  the 
function  itself  was  unimodal . 

The  test  functions  used  were  positive  definite  quadratic  functions. 
Thus,  the  optimum  response  was  the  minimum  objective  function  value. 
The  test  functions  were  constructed  so  that  this  minimum  occurred  at 
values  of  the  input  variables  restricted  to  the  domain  zero  to  one. 
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This  restriction  limited  the  selection  of  initial  starting  points  for 
Powell's  method  to  points  lying  within  this  domain.  Test  points  for  the 
curve-fitting  technique  were  also  restricted  to  the  interval  zero  to  one 
for  all  input  variables. 

To  insure  that  the  quadratic  functions  of  eight  variables  were  in 
fact  positive  definite,  the  following  procedure  was  used  to  generate 
eight  by  eight  matrices  of  quadratic  term  coefficients.  An  eight  by 
eight  matrix  was  generated  and  pre-multi plied  by  its  transpose.  The 
resulting  eight  by  eight  matrix  is  assured  to  be  positive  definite. 

An  explicit  expression  of  all  test  functions  considered  can  be  found 
in  Appendix  B. 

D.  NUMBER  OF  VARIABLES 

Functions  of  two  different  levels  of  input  variables  were  considered 
in  the  investigation.   Initially,  functions  of  three  input  variables 
were  investigated.  The  investigation  concluded  with  a  study  of  functions 
of  eight  input  variables.  While  these  two  cases  are  not  all  inclusive, 
it  was  felt  they  gave  a  comparative  indication  of  how  well  the  two 
techniques  handled  functions  of  low  and  high  levels  of  input  variables. 

E.  NUMBER  OF  OBSERVATIONS 

Again  two  different  levels  of  observations  were  considered.  The  low 
level  was  fixed  at  2n+l  observations.  This  was  the  minimum  number 
required  for  the  orthogonal  design  used  in  the  curve-fitting  technique. 
Twice  the  number  of  observations  in  the  low  level  was  arbitrarily 
decided  upon  for  the  high  level.  An  exception  to  this  high  level  number 
was  made  in  the  case  of  functions  of  three  input  variables.  An  additional 
observation  was  added  to  the  high  level  number  to  allow  the  use  of  a 
composite  factorial  design  with  the  curve-fitting  technique. 
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F.  NUMBER  OF  TRIALS 

Ten  trials  were  conducted  for  each  procedure  that  was  non-deter- 
ministic. These  procedures  included  all  Powell  Searches  and  those  curve- 
fitting  techniques  where  random  choice  was  used  to  locate  some  or  all  of 
the  points. 

G.  MEASURES  OF  EFFECTIVENESS 

Two  measures  of  effectiveness  were  employed  to  compare  results: 

1.  the  mean  optimum  response  attained  by  optimization  technique, 

2.  the  mean  Euclidean  distance  between  the  location  of  the  true 
optimum  and  the  location  of  the  optimum  response  found  by  the 
optimization  technique. 

- 
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III.   PRESENTATION  OF  RESULTS 

Tables  I  through  XI  contain  the  results  of  the  eleven  test  functions 
investigated.  The  following  definition  of  terms  is  provided  to  assist 
in  understanding  the  tabulated  data: 

TECHNIQUE  -  The  name  of  the  optimization  technique  used. 

POWELL  -  direct  search  technique,  Powell's  method  of  conjugate 

directions. 
C.F.  DESIGN  -  curve-fit  technique,  employing  experimental  design 

to  locate  the  test  points. 
C.F.  RANDOM  -  curve-fit  technique,  employing  random  choice  to 

locate  the  test  points. 
75  PERCENT  C.F.  DESIGN  -  curve-fit  technique,  employing  experi- 
mental design  to  locate  the  test  point,  eliminating  the 
worst  25  percent  of  the  test  points. 
50  PERCENT  C.F.  DESIGN  -  curve-fit  technique,  employing  experi- 
mental design  to  locate  the  test  points,  eliminating 
the  worst  50  percent  of  the  test  points. 
75  PERCENT  C.F.  RANDOM  -  curve-fit  technique,  employing  random 
choice  to  locate  the  test  points,  eliminating  the  worst 
25  percent  of  the  test  points. 
50  PERCENT  C.F.  RANDOM  -  curve-fit  technique,  employing  random 
choice  to  locate  the  test  points,  eliminating  the  worst 
50  percent  of  the  test  points. 
#0BS  -  The  number  of  observations  included  in  the  investigation. 
OPTIMUM  RESPONSE  -  The  mean  and  standard  deviation,  v/here  applicable, 
of  the  optimum  response  found  using  the  optimization  technique. 
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The  mean  and  standard  deviation  were  computed  considering  only 
the  results  associated  with  positive  definite  quadratics  gene- 
rated by  the  curve-fitting  technique.  Results  associated  with 
indefinite  quadratic  fitted  surfaces  were  meaningless. 

DISTANCE  -  The  mean  and  standard  deviation,  where  applicable,  of  the 
Euclidean  distance  between  the  location  of  the  true  optimum  and 
the  location  of  the  optimum  response  using  the  optimization 
technique.  The  mean  and  standard  deviation  were  computed  con- 
sidering only  the  results  associated  with  positive  definite 
quadratics  generated  by  the  curve-fitting  technique.  Results 
associated  with  indefinite  quadratic  fitted  surfaces  were 
meaningless. 

REMARKS  -  This  column  contains  the  number  of  indefinite  quadratic 
surfaces  generated  using  the  different  curve-fitting  techniques. 
In  addition,  curve-fitting  techniques  that  yielded  deterministic 
results  are  noted  by  the  statement,  1  trial,  in  this  column. 
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TABLE  I 

3  VARIABLES 

QUADRATIC  I 

TRUE 

OPTIMUM  =  -7. 

9875 

TECHNIQUE 

#OBS 

OPTIMUM 
MEAN 

RESPONSE 
STD.  DEV. 

DISTANCE 
MEAN     STD.  DEV. 

REMARKS 

POWELL 

7 

-7.3566 

0.526 

0.381 

0.273 

15 

-7.5756 

0.277 

0.294 

0.218 

C.F. 
DESIGN 

7 
15 

-7.625 
-7.625 

-- 

0.170 
0.170 

-  - 

1  trial 
1  trial 

C.F. 

RANDOM 

7 

14.9451 

31.03 

1.424 

1.369 

7  indef 
quad 

15 

1.6504 

24.01 

0.608 

1.026 

2  indef 
quad 

75%  C.F. 
DESIGN 

15 

-- 

-- 

-- 

-- 

1  trial 
indef 
quad 

50%  C.F, 
DESIGN 

75%  C.F, 
RANDOM 

50%  C.F, 

RANDOM 


15   -7.5283 


15   -7.2643     0.409 


15   -5.8791     2.569 


0.116 


0.297     0.074 


0.551     0.570 


1  trial 


7  indef 
quad 

7  indef 
quad 


NOTES: 
1. 

2. 


C.F.  design  technique  achieved  the  best  result.  Of  interest  is 
the  fact  that  increasing  the  number  of  observations  did  not 
improve  the  results. 

C.F.  random  technique  was  quite  erratic.  An  improvement  accomp- 
anied an  increase  in  the  number  of  observations.  However,  when 
eliminating  the  worst  25%  (50%)  of  the  test  points  an  improve- 
ment in  mean  optimum  response  and  distance  was  accompanied  by 
an  increase  in  the  number  of  indefinite  quadratics. 
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TABLE  II 

3  VARIABLES 

QUADRATIC  2 

TRUE  OPTIMUM  =  - 

•59.90 

TECHNIQUE 

#OBS 

OPTIMUM 
MEAN 

RESPONSE 
STD  DEV. 

DISTANCE 
MEAN     STD.  DEV. 

REMARKS 

POWELL 

7 

-58.52 

1.02 

0.485 

0.167 

15 

-58.76 

0.816 

0.449 

0.169 

C.F. 
DESIGN 

7 
15 

-58.14 
-58.14 

— 

0.290 
0.290 

— 

1  trial 
1  trial 

C.F. 

RANDOM 

7 

-23.15 

35.73 

0.896 

0.365 

8  indef 
quad 

15 

-59.03 

0.657 

0.274 

0.074 

6  indef 
quad 

75%  C.F, 
DESIGN 

50%  C.F. 
DESIGN 

75%  C.F, 
RANDOM 

50%  C.F, 
RANDOM 


15  7.77 

15  210.60 

15  -39.82 

15  161.83           219.68 


0.919 
2.529 
0.885 
1.619 


1.235 


1  trial 


1  trial 


9  indef 
quad 

8  indef 
quad 


NOTES: 
1. 

2. 
3. 


Powell's  method  achieved  the  best  results  at  both  7  and  15 
observations. 

At  15  observations  the  C.F.  random  techniques  achieved  better 
results  than  Powell's  method.  However,  6  out  of  10  quadratics 
generated  using  this  technique  were  indefinite. 

Eliminating  the  worst  25%  (50%)  of  the  test  points  did  not  re- 
flect any  improvement  using  the  C.F.  design  technique.  Although 
the  mean  optimum  response  and  distance  improved  when  eliminating 
the  worst  25%  of  the  test  points  using  the  C.F.  random  technique, 
the  improvement  was  accomplished  at  the  expense  of  3  more 
indefinite  quadratics  being  generated. 
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TABLE  III 
3  VARIABLES 
EXPONENTIAL  1 
TRUE  OPTIMUM  =  4.476  x  10 


-3 


OPTIMUM  RESPONSE 

DISTANCE 

TECHNIQUE 

#OBS 

MEAN 

STD.  DEV. 

MEAN 

STD.  DEV. 

REMARKS 

POWELL 

7 

6.599xl0'3 

1.71xl0"3 

0.478 

0.165 

15 

5.251xl0"3 

0.43xl0'3 

0.336 

0.111 

C.F. 

7 

6.307xl0"3 



0.366 

__ 

1  trial 

DESIGN 

57.487xl0"3 

15 

-- 

0.766 

-- 

1  trial 

C.F. 

7 

5.491xl0"3 



0.337 



9  indef 

RANDOM 

quad 

15 

8.018xl0"3 

2.70xlO"3 

0.558  ■ 

0.268 

7  indef 
quad 

75%  C.F. 

15 

__ 

__ 

__ 

_  _ 

1  trial 

DESIGN 

indef 
quad 

%  C.F, 
DESIGN 

75%  C.F 
RANDOM 

50%  C.F, 
RANDOM 


15   5.241x10 


-3 


0.389 


15   1056. C26   1829.075     0.746 


0.563 


15  93.852xl0"3  145.24xl0-3  0.506     0.142 


1  trial 


6  indef 
quad 

6  indef 
quad 


NOTES: 
1 


3. 


C.F.  design  technique  achieved  the  best  results  at  7  observa- 
tions and  the  50%  C.F.  design  technique  proved  to  be  the  best 
at  15  observations. 

At  7  observations  C.F.  random  technique  achieved  a  better  mean 
optimum  response  than  C.F.  design  technique.  However,  9  out  of 
10  quadratics  generated  were  indefinite. 

Eliminating  the  worst  25%  (50%)  of  the  test  points  using  the 
C.F.  random  technique  decreased  the  number  of  indefinite  quad- 
ratics generated.  However,  the  mean  optimum  response  in  both 
cases  became  worse. 


TECHNIQUE 
POWELL 


C.F. 
DESIGN 


C.F. 

RANDOM 


75%  C.F. 
DESIGN 

50%  C.F. 
DESIGN 

75%  C.F. 
RANDOM 

50%  C.F. 
RANDOM 


#OBS  MEAN 

7  0.0380 

15  0.0285 

7  0.0306 

15  0.2548 

7  0.0368 

15  0.0943 

15  7.0428 

15  0.0380 


TABLE  IV 

3  VARIABLES 

EXPONENTIAL  2 

TRUE  OPTIMUM  =  0. 

0212 

MUM  RESPONSE 

DISTANCE 

STD.  DEV. 

MEAN 

STD.  DEV. 

REMARKS 

i     0.0154 

0.606 

0.200 

0.0088 

0.408 

0.249 

0.374 

-- 

1  trial 

0.776 

-- 

1  trial 

;     0.0092 

0.694 

0.238 

6  indef 
quad 

0.0652 

0.641 

0.149 

6  indef 
quad 

1.128 


0.365 


15   1.22xl07   1.22xl07     1.802     1.026 


15   106.38     148.94 


0.729     0.302 


1  trial 


1  trial 


8  indef 
quad 

7  indef 
quad 


NOTES: 
1. 


C.F.  design  technique  was  best  at  7  observations.  Powell's 
method  achieved  the  best  results  at  15  observations. 

C.F.  random  techniques  generated  a  large  number  of  indefinite 
quadratics.  Eliminating  the  worst  25%  (50%)  of  the  test  points 
neither  improved  the  mean  optimum  response  or  distance,  nor 
did  it  decrease  the  number  of  indefinite  quadratics. 
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DESIGN 

75%  C.F, 
RANDOM 

50%  C.F 

RANDOM 


TABLE  V 
3  VARIABLES 
EXPONENTIAL  3 
TRUE  OPTIMUM  =  0.2157 


OPTIMUM  RESPONSE 


TECHNIQUE 

#OBS 

MEAN 

STD.  D 

POWELL 

7 

0.5255 

0.313 

15 

0.2969 

0.056 

C.F. 
DESIGN 

7 
15 

0.2232 
0.3423 

-- 

C.F. 

RANDOM 

7 

>1076 

>1076 

15 

0.2873 

0.045 

75%  C.F. 
DESIGN 

15 

0.4668 

-- 

50%  C.F. 

15 

0.2205 

__ 

15   23.7196    46.698 


15   0.3059 


0.073 


MEAN 

0.534 

0.336 

0.155 

0.464 

2.032 

0.357 


DISTANCE 

STD.  DEV 


0.205 
0.110 


2.252 


0.126 


REMARKS 


1  trial 

1  trial 

7  indef 
quad 

6  indef 
quad 


0.662 

— 

1  trial 

0.164 

-- 

1  trial 

0.625 

0.475 

5  indef 
quad 

0.330 

0.054 

7  indef 
quad 

NOTES: 
1. 

2. 


C.F.  design  technique  achieved  the  best  results  at  7  observa- 
tions. At  15  observations,  C.F.  design  eliminating  the  worst 
50%  of  the  test  points  achieved  the  best  results. 

C.F.  random  techniques  were  extremely  erratic  generating  many 
indefinite  quadratics.  In  the  case  of  7  observations,  one  bad 
fit  caused  the  mean  optimum  response  to  become  so  large  as  to 
be  meaningless. 
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TABLE  VI 

8  VARIABLES 

QUADRATIC  1 

TRUE 

:  OPTIMUM  =  -44.61 

TECHNIQUE 

#OBS 

OPTIMUM 
MEAN 

RESPONSE 
STD.DEV. 

DISTANCE 
MEAN     STD.  DEV. 

REMARKS 

POWELL 

17 

-41.97 

1.207 

1.028 

0.047 

34 

-43.50 

0.591 

0.962 

0.047 

C.F. 
DESIGN 

17 
34 

-44.18 
-27.26 

21.14 

0.644 
1.871 

1.992 

1  trial 

2  indef 
quad 

C.F. 

RANDOM 

17 

— 

-- 

-- 

-- 

10  indef 
quad 

34 

-34.30 

7.76 

1.584 

1.844 

8  indef 
quad 

75%  C.F. 
DESIGN 

34 

-43.45 

1.569 

0.55 

0.066 

7  indef 
quad 

50%  C.F. 
DESIGN 

34 

-43.14 

-- 

0.827 

— 

9  indef 
quad 

75%  C.F. 
RANDOM 

34 

— 

-- 

-- 

-- 

10  indef 
quad 

50%  C.F. 
RANDOM 

34 

-- 

— 

-- 

-- 

10  indef 
quad 

NOTES: 

1.  C.F. 

desic 

in  achieved 

the  best  resi 

jits  at 

17 

observat' 

ions.  At 

34  observations,  Pov/ell's  method  proved  to  be  the  best 
technique.  However,  it  is  significant  to  note  that  the 
results  achieved  using  C.F.  design  techniques  with  17 
observations  is  better  than  that  achieved  by  Powell  with  34 
observations. 

C.F.  random  techniques  were  a  complete  disaster  generating 
all  indefinite  quadratic  surfaces  in  3  out  of  4  investigations 
used. 
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TECHNIQUE 
POWELL 


C.F. 

DESIGN 


C.F. 

RANDOM 


75%  C.F. 
DESIGN 

50%  C.F. 
DESIGN 

75%  C.F. 
RANDOM 

50%  C.F. 
RANDOM 


#OBS 
17 
34 
17 
34 


TABLE  VII 
8  VARIABLES 
QUADRATIC  2 
TRUE  OPTIMUM  =  -91.89 


OPTIMUM  RESPONSE 
MEAN     STD.  DEV. 


-79.56 

-85.04 

15.31 

-16.92 


9.53 
4.02 

49.88 


17 

34   676.47 

34   1358.07    1308.57 

34 

34 

34 


DISTANCE 
MEAN     STD. DEV 


0.997 
0.942 
0.763 
1.099 


0.212 
0.233 

1.314 


3.751 


3.153     1.885 


REMARKS 


1  trial 

6  indef 
quad 

10  indef 
quad 

9  indef 
quad 

8  indef 
quad 

10  indef 
quad 

10  indef 
quad 

10  indef 
quad 


NOTES: 
1. 

2. 


Powell's  method  achieved  the  best  results  at  both  17  and  34 
observations. 

All  curve-fitting  techniques  were  inefficient.  A  probable 
cause  of  this  problem  was  the  fact  that  the  quadratic  function 
investigated  here  had  sharply  rising  contours  in  the  vicinity 
of  the  optimum. 
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TABLE  VIII 

8  VARIABLES 

QUADRATIC  3 

TRUE 

OPTIMUM  =  -68 

.25 

TECHNIQUE 

#OBS 

OPTIMUM 
MEAN 

RESPONSE 
STD.  DEV. 

DISTANCE 
MEAN     STD.  DEV. 

REMARKS 

POWELL 

17 

-43.29 

30.42 

1.201 

0.454 

34 

-47.67 

26.62 

1.135 

0.448 

C.F. 
DESIGN 

17 
34 

154.19 
1681.1 

3398.9 

1.011 
14.68 

31.56 

1  trial 

3  indef 
quad 

C  .F. 

RANDOM 

17 

— 

— 

-- 

-- 

10  indef 
quad 

34 

— 

— 

— 

-- 

10  indef 
.  quad 

75%  C.F. 
DESIGN 

34 

1617.68 

1282.98 

2.893 

1.495 

7  indef 
quad 

50%  C.F. 
DESIGN 

34 

-- 

— 

— 

-- 

10  indef 
quad 

75%  C.F. 
RANDOM 

34 

— 

-- 

— 

-- 

10  indef 
quad 

50%  C.F. 
RANDOM 

34 

-- 

— 

— 

— 

10  indef 
quad 

NOTES: 
1. 


Powell's  method  achieved  the  best  results  at  both  17  and  34 
observations . 

All  curve-fitting  techniques  were  inefficient.  A  probable 
cause  of  this  problem  v/as  the  fact  that  the  quadratic  function 
investigated  here  had  sharply  rising  contours  in  the  vicinity 
of  the  optimum. 
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TECHNIQUE 
POWELL 


C.F. 

DESIGN 


C.F. 

RANDOM 


NOTES 


1. 


TABLE  IX 
8  VARIABLES 
EXPONENTIAL  1 
TRUE  OPTIMUM  =  1.3x10 


-9 


OPTIMUM  RESPONSE 
#OBS   MEAN     STD.  DEV 


DISTANCE 
MEAN     STD.  DEV 


17  1245.08xl0~93605.4xl0~9  1.072 
34    8.46xl0'9  18.38xl0"9  0.925 


0.214 
0.188 


17 


2.3x10" 


34    10.6x10 


17 


-9 


0.526 
0.674 


34 


75%  C.F. 
DESIGN 

34 

81.93x1 

50%  C.F. 
DESIGN 

34 

3.31x10 

75%  C.F. 
RANDOM 

34 

-- 

50%  C.F. 
RANDOM 

34 

-- 

•9  ,„>  nr   ,rt-9 


-9 


0.542 


REMARKS 


2. 


1  trial 

9  indef 
quad 

10  indef 
quad 

10  indef 
quad 

7  indef 
quad 

9  indef 
quad 

10  indef 
quad 

10  indef 
quad 


C.F.  design  technique  achieved  the  best  results  at  17 
observations  while  Powell's  method  proved  best  at  34  observa- 
tions. However,  the  results  at  17  observations  using  C.F. 
design  were  better  than  those  using  Powell's  method  with  34 
observations. 

At  34  observations,  C.F.  design  eliminating  the  worst  50%  of 
the  test  points  achieved  results  superior  to  Powell's  method. 
However,  9  out  of  the  10  quadratics  generated  were  indefinite 

C.F.  random  techniques  were  completely  inefficient  generating 
indefinite  quadratic  surfaces  in  every  situation  they  were 
used. 
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TABLE  X 

8  VARIABLES 

EXPONENTIAL  2 

TRUE  OPTIMUM  =  1.02xlO"25 

OPTIMUM  RESPONSE        DISTANCE 

TECHNIQUE 

#OBS 

MEAN     STD.  DEV.    MEAN     STD.  DEV. 

REMARKS 

POWELL 

17 

61.27xlO"25  108.13xl0'25  1.096     0.190 

34 

14.40xl0"25  23.61xl0"25  0.949     0.204 

C.F. 

17 

33.4xl0"25    --        0.591 

1  trial 

DESIGN 

34 

__        __         __       _ _ 

10  indef 
quad 

C.F. 

17 

_  _        -_         _ _       -  - 

10  indef 

RANDOM 

quad 

34 

-- 

10  indef 
quad 

75%  C.F. 

34 

18599xl0"25  26296xl0"25  1.262     0.495 

7  indef 

DESIGN 

quad 

50%  C.F. 

34 

94.3xl0"25    —        0.696 

9  indef 

DESIGN 

quad 

75%  C.F. 

34 

_  —        —  —        _  _       _  _ 

10  indef 

RANDOM 

quad 

50%  C.F. 

34 

—  _        — —         _ _       — _ 

10  indef 

RANDOM 

quad 

NOTES: 

1.  At 

17  obs 

ervations  C.F.  design  achieved  the  best  result 

s  while 

Powell's  method  was  superior  at  34  observations. 

C.F.  random  techniques  were  completely  inefficient  generating 
indefinite  quadratic  surfaces  in  every   situation  they  were  used 
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TECHNIQUE 
POWELL 


C.F. 
DESIGN 


C.F. 

RANDOM 


75%  C.F, 
DESIGN 

50%  C.F, 
DESIGN 

75%  C.F, 
RANDOM 

50%  C.F 
RANDOM 

NOTES: 


TABLE  XI 
8  VARIABLES 
EXPONENTIAL  3 
TRUE  OPTIMUM  =  9.91x10 


-6 


OPTIMUM  RESPONSE 
#OBS   MEAN     STD.  DEV. 

17  67.95xl0"6  50.18xl0"6 

34  17.0xl0"6   5.11xl0"6 


>10 
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DISTANCE 
MEAN     STD.  DEV 

1.078  0.269 

0.960  0.277 

0.516 

1.912  1.903 


17  12.92x10' 
34   >1026 

17 


34  1172.35xl0'61147.92xl0"6  1.003     0.349 


34   21.74xlO"6  12.09xl0"6  0.608     0.104 


34   13.63xlO~6  0.836xl0~6  0.540     0.025 


34     10' 


34 


2.482 


REMARKS 


1  trial 

4  indef 
quad 

10  indef 
quad 

8  indef 
.  quad 

2  indef 
quad 

6  indef 
quad 

9  indef 
quad 

10  indef 
quad 


C.F.  design  achieved  the  best  results  at  17  observations. 
Although  Powell's  method  proved  to  be  the  best  at  34  observations 
the  results  achieved  were  not  as  good  as  those  achieved  using 
C.F.  design  with  17  observations. 

At  34  observations,  C.F.  design  eliminating  the  worst  25%  (50%) 
of  the  test  points  improved  the  mean  optimum  response  and  dist- 
ance. However,  when  eliminating  the  worst  25%  a  decrease  in  the 
number  of  indefinite  quadratics  occurred,  while  elimination  of 
the  worst  50%  caused  an  increase  in  the  number  of  indefinite 
quadratics. 

C.F.  random  techniques  were  inefficient  due  mainly  to  the 
large  number  of  indefinite  quadratics  generated. 
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IV.  CONCLUSIONS 

Overall,  the  direct  search  technique,  Powell's  method  of  conjugate 
directions,  proved  to  be  the  best  technique  of  those  considered  in  the 
investigations.  A  particularly  desirable  feature  of  Powell's  method  was 
the  fact  that  results  achieved  using  this  technique  converged  to  the  true 
optimum  response  when  increasing  the  number  of  observations.  Thus, 
Powell's  method  exhibited  the  characteristic  that  the  more  information 
(function  evaluations)  available,  the  better  the  results  that  could  be 
expected  to  be  achieved.  A  decrease  in  the  distance  between  the  loca- 
tion of  the  true  optimum  response  and  the  location  of  the  optimum  found 
using  Powell's  method  was  also  experienced  as  the  number  of  observations 
was  increased. 

The  curve-fitting  techniques,  as  a  whole,  were  quite  erratic.  The 
biggest  single  problem  area  was  the  fact  that  the  curve-fitting  tech- 
niques generated  a  large  number  of  indefinite  quadratic  surfaces.  This 
situation  was  prevalent  in  both  the  case  where  the  test  points  were 
located  by  random  choice  and  the  case  where  experimental  design  was 
used  to  locate  the  test  points.  However,  it  should  be  noted  that  the 
problem  of  indefinite  quadratics  arose  in  the  curve-fitting  techniques 
employing  experimental  design  only  in  those  cases  where  the  total  number 
of  observations  exceeded  that  number  required  by  the  experimental  design. 
In  this  situation,  the  remaining  number  of  test  points  were  located  by 
random  choice.  Thus,  the  results  achieved  strongly  indicate  a  high 
correlation  between  the  generation  of  indefinite  quadratic  surfaces 
and  the  use  of  random  choice  to  locate  any  or  all  of  the  test  points. 
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A  second  problem  encountered  when  using  curve-fitting  techniques 
was  the  trend  toward  higher  standard  deviations  of  the  optimum  responses. 
This  situation  resulted  from  the  fact  that  the  curve-fitting  techniques 
generated  many  bad  fits  with  respect  to  the  optimum  response.  This 
problem  coupled  with  a  reduction  in  the  sample  size  due  to  the  genera- 
tion of  indefinite  quadratics  led  to  a  higher  variance  of  optimum 
response,  thus  a  high  standard  deviation. 

To  overcome  these  problems  the  effect  of  reducing  the  number  of 
observations  considered  by  the  curve-fitting  technique  by  elimination 
of  the  worst  25%  and  50%  of  the  test  points  was  investigated.  This 
approach  did  not  yield  consistent  results.  In  some  cases  the  number  of 
indefinite  quadratics  generated  was  reduced,  while  in  other  cases  the 
number  remained  the  same  or  increased.  In  addition,  in  many  cases  where 
the  number  of  indefinite  quadratics  was  reduced,  an  increase  in  the 
number  of  bad  fits  with  respect  to  the  optimum  response  was  experienced. 

However,  curve-fitting" techniques  do  merit  consideration  under 
certain  circumstances.  Provided  that  the  objective  function  did  not 
have  sharply  rising  contours  in  the  immediate  location  of  the  true 
optimum,  curve-fitting  techniques  employing  experimental  design,  where 
the  number  of  test  points  considered  was  exactly  equal  to  the  number 
required  by  the  experimental  design,  provided  results  better  than  those 
achieved  by  Powell's  method.  In  fact  for  some  of  the  test  functions 
investigated,  the  results  achieved  using  the  curve-fitting  techniques 
employing  experimental  design  were  better  than  those  achieved  using 
Powell's  method  with  twice  the  number  of  observations. 

Thus,  on  the  whole  Powell's  method  is  an  efficient  method  of 
optimization  that  consistently  achieved  better  results  than  any   of  the 
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curve-fitting  techniques  investigated.  Under  certain  circumstances, 
the  curve-fitting  technique  employing  experimental  design  to  locate 
test  points  provided  better  results  and  was  more  efficient  than  Powell's 
method. 
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V.   SUGGESTED  FUTURE  INVESTIGATIONS 

The  problem  of  generating  indefinite  quadratic  fits  when  using 
curve-fitting  techniques  was  never  overcome.  In  future  investigations 
one  way  of  addressing  this  problem  would  be  to  constrain  the  regression 
procedures  so  that  the  coefficients  of  the  quadratic  terms  are  all 
positive.  Another  method  that  could  be  used,  either  in  connection  with 
the  above  technique  or  by  itself,  is  to  force  the  fitted  surface 
through  a  certain  point.  As  a  suggestion  it  might  prove  useful  to 
force  the  fitted  surface  through  the  best  of  the  test  points.  The  best 
test  point  being  chosen  as  the  one  with  the  smallest  function  value  in 
the  case  of  a  minimization  problem. 

Another  interesting  question  that  could  be  considered  is  whether 
a  combination  of  the  search  and  curve-fitting  procedures  would  achieve 
better  results  than  either  procedure  achieved  by  itself.  A  suggested 
approach  to  this  question  would  be  to  use  Powell's  search  initially  to 
find  the  general  area  of  the  optimum  solution.  Once  this  has  been 
achieved,  the  curve-fit  design  procedure  could  be  used  in  this  general 
area  to  obtain  the  optimum  solution. 
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APPENDIX  A 
CURVE-FITTING  PROGRAM 

The  purpose  of  the  curve-fitting  program  was  threefold: 

1)  determine  the  location  of  the  test  points  and  their  respective 
functional  values, 

2)  fit  a  quadratic  surface  to  these  test  points, 

3)  determine  the  location  of  the  optimum  response  of  the  resulting 
"fitted"  quadratic  surface  and  compute  the  true  functional 
value  at  that  point. 

A.  LOCATION  OF  TEST  POINTS 

The  program  considered  two  different  methods  for  locating  test 
points  in  the  input  variable  space,  random  choice  or  placement  in 
accordance  with  an  experimental  design.  Input  data  to  the  program  pro- 
vided the  basis  for  selection  of  the  appropriate  method.  A  uniform 
random  number  generator  was  used  in  connection  with  the  method  of 
random  choice.  The  two  experimental  designs  used,  composite  factorial 
and  orthogonal,  were  written  into  the  program  to  be  constructed  by  the 
computer  as  they  were  needed. 

Upon  selection  of  the  appropriate  method  a  matrix  v/as  constructed, 
each  row  of  the  matrix  containing  one  observation  of  the  input  variables. 
At  this  stage  the  number  of  columns  in  the  matrix  was  equal  to  the  number 
of  input  variables.  The  size  of  the  matrix  was  then  expanded  in  the 
following  manner.  For  each  observation  the  value  of  each  input  variable 
was  squared  and  stored  in  the  corresponding  row  of  the  matrix,  thus, 
doubling  the  number  of  columns  of  the  matrix.  A  final  column  was  added 
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to  the  matrix  by  computing  the  function  value  corresponding  to  each 
observation  of  the  input  variables. 

The  completed  matrix  was  then  input  to  the  curve-fitting  procedure 
unless  a  reduction  in  the  number  of  observations  was  called  for.  In 
that  case  the  function  values  were  scanned  by  the  computer  to  determine 
the  worst  25%  or  50%  of  the  values  as  appropriate.  These  worst  function 
values  along  with  their  corresponding  input  variables  and  their  squares 
were  eliminated  from  the  matrix  reducing  the  number  of  rows  in  the 
matrix  input  to  the  curve-fitting  procedure. 

B.  FITTING  THE  QUADRATIC  SURFACE 

Multiple  linear  regression  techniques  were  used  to  fit  a  quadratic 
surface  to  the  test  points.  This  section  of  the  program  consisted  of 
three  subroutines  and  the  procedures  for  computation  of  the  regression 
coefficients.  The  three  subroutines  used  were  CORRE,  ORDER,  and  MINV. 
These  subroutines  are  included  in  the  IBM  360  Scientific  Subroutine 
Package  and  are  stored  internally  on  the  computer. 

The  purpose  of  subroutine  CORRE  was  to  compute  the  means,  the 
standard  deviations  and  a  matrix  of  correlation  coefficients  of  the 
variables  included  in  the  regression.  In  this  case,  the  original  input 
variables,  their  squares  and  the  function  values.  The  matrix  constructed 
in  the  initial  section  of  the  program  served  as  input  to  CORRE  to  accomp- 
lish these  tasks. 

The  resulting  correlation  coefficient  matrix  from  subroutine  CORRE 
was  then  input  to  subroutine  ORDER.  The  function  of  subroutine  ORDER 
was  to  compute  a  matrix  containing  the  correlation  coefficients  of  the 
independent  variables  and  a  vector  containing  the  correlation  coefficients 
of  the  independent  variables  with  the  dependent  variables.  In  the 
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program,  the  original  input  variables  and  their  squares  were  designated 
the  independent  variables,  and  the  function  value  was  designated  the 
dependent  variable. 

To  check  the  validity  of  the  regression  the  matrix  of  correlation 
coefficients  among  independent  variables  calculated  by  subroutine  ORDER 
was  input  to  subroutine  MINV.  Subroutine  MINV  computed  the  determinant 
of  this  matrix.  If  the  determinant  was  non-zero,  the  regression  was 
valid  and  the  program  continued.  However,  if  the  determinant  was  zero 
the  program  terminated  due  to  the  presence  of  multi-col  linearity  in  the 
regression. 

Provided  the  regression  was  valid,  the  next  step  was  to  compute  the 
regression  coefficients  of  the  independent  variables.  To  accomplish 
this  task  a  portion  of  subroutine  MULTR,  also  included  in  the  IBM  360 
Scientific  Subroutine  Package,  was  used.  The  standard  deviations  com- 
puted by  subroutine  CORRE,  and  the  matrix  of  correlation  coefficients 
among  the  independent  variables  and  the  vector  of  correlation 
coefficients  between  the  independent  variables  and  the  dependent 
variable  computed  by  subroutine  ORDER  served  as  input  to  this  subroutine, 
The  output  of  the  subroutine  was  a  vector  containing  the  regression 
coefficients. 

C.  LOCATION  OF  OPTIMUM  RESPONSE 

The  vector  of  regression  coefficients  contained  the  necessary 
information  to  determine  the  location  of  the  optimum  response  of  the 
fitted  surface.  The  length  of  this  vector  was  2n,  where  n  is  the  number 
of  input  variables.  The  resulting  fitted  quadratic  was  of  the  form: 

n         n     2 
f  =  z  a.  x.  +  z  b.  x.  +  constant 
i=l  n  1   i=l  ] 
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The  value  of  the  a.'s  was  contained  in  the  first  n  elements  of  the  vector 
of  regression  coefficients  and  the  value  of  the  b- 's  was  contained  in 
the  last  n  elements. 

The  location  of  the  optimum  (minimum)  response  was  determined  as 

follows.  The  values: 

1  a ' 
x  •  -  ~  ■p"  7"   j  i    i  ,  .  .  .  ,  n , 

were  computed.  These  values  uniquely  determined  the  location  of  the 
minimum  response  of  the  fitted  quadratic  provided  that  the  quadratic 
was  positive  definite.  A  vector  containing  the  location  of  the  optimum 
response  of  the  fitted  surface  was  then  substituted  into  the  true 
objective  function  to  determine  the  optimum  objective  response  for  the 
curve-fitting  procedures. 
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APPENDIX  B 
TEST  FUNCTIONS 

I.  THREE  VARIABLE  FUNCTIONS 


A. 


QUADRATIC  1 
F  =  XT 


2  1  2 

4  3 

6 


X  +  X 


True  Optimum  =  -7.9875 


-6 

-8 

-13 


At  X,  =  0, 
x2  =  0, 

x3  =  o, 


625 

4 

725 


QUADRATIC  2 
F-XT 


29 


26 

30] 

29 

34 

45 

X  +  X 


-80.2 
-81.0 

-98.4 


True  Optimum  =  -59.90 


At  X]  =  0 
x2  =  0 
x3  =  o 


EXPONENTIAL  1 


F  =  exp/  X 


( 


T 


X  +  X 


-10 

-  6 

-  8J 


True  Optimum  =  0.004476 


At  X1  =  0 
x2  =  0 
x3  =  o 


2727 
8636 
3636 


D.  EXPONENTIAL  2 


F  =  exp/X 


-3 

-2 

4 


X  +  X 


T 


-9 

-3 
1 


True  Optimum  =  0.0212 


At  X1  =  0 

x2  =  0 
x3  =  o 


,6364 
,9318 
,8182 
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EXPONENTIAL  3 


F  =  exp  /  X 


T 


8  2-3 

2  -2 

4 


X  +  X 


-3 
2 


True  Optimum  =  0.2157 


At  X]  =  0.2273 
X2  =  0.8864 
X3  =  0.3636 


II.  EIGHT  VARIABLE  FUNCTIONS 


A.  QUADRATIC  1 


F  =  X 


T  n 


2  4 
0  1 


5 

0 

11 


2 

0 

2 

0 

4 

2 

1 

1 

6 

0 

0 

0 

12 

0 

5 

X  +  X 


T 


True 

Optimum 

=  -44.61 

At 

Xl 

=  0. 

6 

X2 

=  0.9 

X3 

=  0.2 

x4 

=  0.8 

QUADRATIC  2 

F  =  XT 

93  -53 

-54  32 

13 

31 

-67 

43~ 

103 

54  -40 

-28 

1 

46 

-1 

50  -28 

-1 

-20 

54 

-30 

22 

8 

7 

-30 

14 

32 

-11 

25 

3 

-30 

69 

-15 

30 

-45 

- 

57 

True  Optimum  : 

=  -91.89 

At 

Xl 
X2 
X3 

=  0.7 
=  0.7 
=  0.4 

x„ 

=  0. 

3 

-18.8 
-13.6 
-38.0 
-24.4 
-25.8 
-23.8 
-24.8 
-10.4 


Xo  = 


X  +  X 


0.7 
0.2 
0.3 
0.8 


T  r 


X6  = 
X7  = 

Xo  = 


-96.2 

-75.0 
6.4 
3.2 
4.0 

-65.8 
48.0 

-89.4 


0.6 
0.8 
0.2 
0.3 


36 


QUADRATIC 

3 

F  =  XT 

43 

-23  16 

-29 

25 

-22  22  -21 

50  -22 

53 

-45 

39  -39   37 

19 

-28 

24 

-21  21  -20 

106 

-69 
76 

60  -60   57 

-45  45  -43 

41  -40   37 

45  -35 

_ 

38_ 

True  Optimum 

=  -68.25 

At  X]  =  0.7 
x2  =  0.1 
X3=0.2 

X„  =  0.4 

X  +  X 


-  1.0 

-  78.0 
32.8 

-146.2 
92.0 

-  85.2 
79.6 

-  83. Oj 


=  0.1 
=  0.6 
=  0.2 
=  0.6 


EXPONENTIAL 

/ 
F  =  exp 


-3 
-2 

4 


0 

0 

0 

10 


V 


True  Optimum  =  1.3  x  10" 


0 
0 
0 

-4 
0 

-2 
8 


At  X1   = 

h- 
U  = 


X  +  X 


0.9 
0.7 
0.3 
0,9 


T    r 


-11.4 

^ 

-  7.6 

8.0 

-16.4 

-15.0 

-  7.4 

1.2 

-  2.0 

/ 

X5  = 
X6  = 


0.7 

0.3 
0.6 
0.4 


EXPONENTIAL  2 

/  T 

F  =  exp     1/144  X1 


V 


806  374  473  390  456  388  570  620 

395  453  417  585   275  405  452 

587  529  696  417   583  535 

810  574  297  403  589 

1581    411    644  624 

385  591    372 

1574  693 

930 


X  +  X 


t  r 


■25.84 
-20.95 
•27.40 
•25.00 
•34.75 
■21.61 
•42.48 
■32.72 


^i 
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True  Optimum  =  1.02  x  10 


25 


At  X]  = 
h- 

X„  = 


0.1032 

X5 

=  0.3980 

0.1502 

X6 

=  0.7504 

0.1759 

h 

=  0.9039 

0.3889 

Xo 

=  0.8039 

F. 


EXPONENTIAL 


F  ■  exp 


1/144  X 


T 


295 


171 
687 


-170 

-  90 

316 


V 

True  Optimum  =  9.91  x  10 


-6 


123 

136 

-140 

302 


125 
-138 
-150 
-  42 

329 


At  X. 


128 
89 
28 
149 
199 
687 


5- 
138- 
196- 
-192 
-108 
-125 
707 


X,  = 


0.4004 
0.4158 
0.6355 
0.7187 


24 

X  +  XT 

-4.48 

N 

191 

-6.76 

44 

-1.63 

14 

-2.27 

114 

-1.28 

122 

-10.40 

42 

-7.53 

269 

-1.84 

/ 

X5  = 


0.4314 
0.7791 
0.8928 
0.2144 
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